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1.  Statement  of  the  Problem  Studied 


This  report  summarizes  efforts  by  McDonnell  Douglas  Aerospace,  with  Summus  Ltd.  (Columbia, 
SC)  and  the  University  of  Maryland  as  subcontractors,  on  an  ARPA  University-Industry  ATR 
(Automatic  Target  Recognition)  Initiative  contract  from  14  November  1994  to  14  February  1997. 
The  principal  program  objectives  of  this  effort  were: 

•  Improved  detection/recognition/identification  of  targets  in  clutter, 

•  Increased  probability  of  acquisition  and  reduced  circular  error  probability  (CEP)  for  autono¬ 
mous  weapons,  and 

•  Fast  automated  reference  preparation  from  reconnaissance  imagery,  made  possible  by  3-D 
site  estimation  from  the  imagery,  and  matching  of  reference  information  and  sensed  image 
data  across  spectrums.  Reference  imagery  is  not  traditionally  used  for  prosecution  of  time 
critical  mobile  targets  because  references  cannot  be  extracted  within  target  time  cycles. 

McDonnell  Douglas  and  Summus,  Ltd.  investigated  a  variety  of  applications  of  multiresolution 
wavelet  analysis  to  improve  the  separation  of  the  target  signature  and  shape  from  background  clutter. 
Wavelet  analyses  are  well-suited  for  efficiently  representing  the  geometric  structure  in  real  imagery, 
as  evidenced  by  the  outstanding  results  achieved  by  wavelet-based  approaches  to  image  compres¬ 
sion.  Planned  approaches  included: 

•  The  development  of  improved  clutter  model  estimation  using  multiresolution  wavelet  analy¬ 
ses.  These  would  be  employed  for  clutter  screening  ,  providing  a  mechanism  to  reduce  false 
alarms  on  background  clutter  regions  which  would  otherwise  yield  a  good  match  to  the  tar¬ 
get. 

•  The  development  of  adaptive  wavelet  transform  techniques  which  are  sensitive  to  specific 
features  or  content  of  an  image.  Examples  include  closed  set  wavelet  transforms  adapted  to 
an  image  by  partitioning  along  detected  edges,  or  the  wavelet  packet  transform  which  yields  a 
richer  set  of  coefficients  from  which  a  “best  basis”  may  be  extracted.  Applications  of  ad^- 
tive  wavelet  transforms  include  segmentation  (useful  in  reference  preparation  from  recon¬ 
naissance  imagery),  texture  analysis,  and  extraction  of  higher  order  statistics  within  parti¬ 
tioned  regions  of  an  image  (potentially  useful  in  the  context  of  the  MDA’s  General  Pattern 
Match  algorithm,  which  is  nominally  based  on  zeroth-order  statistics). 

The  University  of  Maryland  investigated  the  application  of  image  understanding  techniques  to  the 
problem  of  estimating  3-D  geometries  of  target  sites  from  reference  imagery.  This  effort  extended 
site  model  construction  and  maintenance  techniques  originally  developed  under  ARPA’s  Research 
and  Development  of  Image  Understanding  Systems  program.  The  ultimate  objective  of  this  site 
model  effort  was  to  develop  fast  and  automated  techniques  for  constructing  3-D  site  models  of  target 
sites  using  reconnaissance  imagery.  The  resulting  site  models  could  then  be  used  to  provide  two 
alternative  forms  of  reference:  (a)  the  3-D  site  model  itself  or  (b)  a  2-D  scene  rendered  to  the  per¬ 
spective  of  the  weapon  system  approach  on  the  target.  The  goal  of  accelerating  and  automating  the 
site  model  construction  process  was  to  make  target  references  available  quickly  enough  to  be  used 
within  time  cycles  for  time  critical  targets. 
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2.  Summary  of  Important  Results 


2.1  Summus  Ltd. 


2.1.1  Target  /  Clutter  Separation 

Summus  developed  an  algorithm  for  separation  of  targets  from  background  clutter,  using  a  wavelet- 
based  oscillation  measure  which  characterizes  texture  properties.  This  measure  provides  a  clutter 
feature  which  can  be  used  for  target/clutter  segmentation  or  as  a  decision  aid  for  other  algorithms. 
To  provide  a  complete  mechanism  for  target  detection,  Summus  also  developed  a  quadtree  based 
image  segmentation  algorithm.  The  algorithm  partitions  an  input  image  into  homogeneous  regions 
based,  for  example,  on  pixel  intensity.  The  image  segmentation  algorithm  can  be  applied  to  the  im¬ 
age  itself  or  it  can  be  combined  with  the  above  target/clutter  separation  algorithm  for  automatic  tar¬ 
get  recognition. 

Summus  evaluated  the  utility  of  the  wavelet-based  oscillation  measure  for  target  detection,  using  a 
limited  testbed  of  50  FLIR  images  of  mobile  targets  in  both  clutter  and  open  terrain.  The  algorithm 
was  able  to  distinguish  targets  with  a  Pd  of  97%,  with  a  Pfa  of  6%.  In  1996,  Summus  extended  this 
algorithm  to  target  recognition  and  tracking  in  image  sequences  through  an  adaptive  temporal  modi¬ 
fication  of  the  wavelet-based  oscillation  measure.  More  detailed  descriptions  of  these  investigations 
and  the  results  can  be  found  in  [2],  [8],  and  in  forthcoming  manuscripts. 

Summus  implemented  some  of  these  algorithms  on  the  Texas  Instruments  C40  digital  signal  proces¬ 
sor.  These  implementations  were  to  be  used  for  evaluations  in  the  context  of  MDA’s  General  Pat¬ 
tern  Match  hardware,  during  the  optional  second  phase  of  this  program,  which  was  not  funded. 

2.1.2  Feature  Enhancement  and  Extraction 

Sununus  also  investigated  two  different  approaches  to  enhance  image  quality  for  improved  target 
recognition  performance.  The  first  approach  is  based  on  use  of  conventional  highpass  local  neigh¬ 
borhood  filters.  Implementing  the  filters  in  the  wavelet  domain  results  in  equal  or  better  image  en¬ 
hancement  when  compared  to  typical  pixel  domain  based  applications.  There  is  also  a  reduction  in 
computational  time,  presuming  that  the  wavelet  transform  is  already  applied  for  some  other  reason, 
e.g.  compression.  Details  of  this  effort  can  be  found  in  [2]. 

The  second  approach  involves  the  use  of  nonlinear  partial  differential  equation  methods  to  solve  a 
nonlinear  diffusion  equation.  This  approach  essentially  treats  the  current  image  as  a  blurred  version 
of  the  source  image.  By  reversing  the  blurring  (diffusion)  process,  it  is  possible  to  project  back  to 
the  source  image.  This  approach  operates  in  the  pixel  domain  and  is  well  suited  for  large  levels  of 
noise.  It  enhances  features  in  the  images  and  improves  target/clutter  estimation.  A  more  detailed 
description  of  this  investigation  and  the  results  can  be  found  in  [2]. 

In  1996,  Summus  extended  their  earlier  work  with  nonlinear  PDEs  to  demonstrate  effective  edge 
detection  and  segmentation  of  images  in  clutter.  Sunmius  also  used  nonlinear  PDEs  to  enhance  im¬ 
ages  which  had  been  magnified  using  the  wavelet  transform.  For  magnification,  the  scaling  filter  of 
the  wavelet  transform  acts  as  an  interpolating  filter,  which  causes  the  resulting  image  to  be  some- 
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what  blurred.  The  blurring  effect  can  be  reduced  using  nonlinear  PDEs.  A  more  detailed  description 
of  this  investigation  and  the  results  can  be  found  in  [8]. 

Summus  is  now  working  on  a  new  resolution-independent  image  file  format  based  on  wavelets  and 
nonlinear  PDEs.  Nonlinear  PDEs  will  be  used  to  estimate  the  source  image  (i.e.,  the  “real  world” 
scene)  from  which  a  sensed  image  is  derived.  The  image  content  is  modeled  in  terms  of  vectors  of 
infinitely  fine  resolution  in  the  wavelet  domain.  This  format  will  be  well-suited  for  compression  and 
for  image  analysis  (including  target  recognition),  and  will  also  facilitate  efficient  transmission  and 
database  retrieval.  More  details  on  this  effort  can  be  found  in  [8]. 

Additional  details  on  these  efforts  will  be  provided  in  forthcoming  manuscripts. 

2.2  University  of  Maryland 

2.2.1  Site  Model-Based  Image  Exploitation 

The  University  of  Maryland  applied  image  understanding  techniques  (originally  developed  under 
ARPA’s  Research  and  Development  of  Image  Understanding  Systems  program  -  RADIUS)  to  the 
problem  of  estimating  3-D  geometries  of  target  sites  from  reference  imagery.  Using  a  combination 
of  manual  and  automated  techniques,  they  constructed  a  detailed  site  model  of  the  Labadie  power 
plant  site  (based  on  target  database  information  provided  by  MDA)  using  approximate  platform  ori¬ 
entation  parameters,  and  registered  two  optical  image  sequences  to  the  Labadie  site.  They  carried 
out  image  resection  using  a  set  of  identified  control  points,  and  have  constructed  a  site  model  in¬ 
cluding  the  main  buildings  and  functional  areas.  Their  approach  uses  constrained  optimization  tech¬ 
niques  for  the  semi-automated  construction  of  prototypical  objects.  A  detailed  report  of  this  effort 
can  be  found  in  [1]. 

The  ultimate  objective  of  this  site  model  effort  was  to  develop  fast  and  automated  techniques  for 
constructing  3-D  site  models  of  target  sites  using  reconnaissance  imagery.  The  site  model  could  then 
be  used  to  provide  two  alternative  forms  of  reference:  (a)  the  3-D  site  model  itself  or  (b)  a  2-D  scene 
rendered  to  the  perspective  of  the  weapon  system  approach  on  the  target.  These  forms  of  reference 
were  to  be  evaluated  in  the  context  of  MDA’s  General  Pattern  Match  algorithm  during  the  optional 
second  phase  of  this  program,  which  was  not  funded. 

2.2.2  Dynamic  Re2istration  of  Sensor  Imagery  to  Site  Models 

The  University  of  Maryland  developed  methods  for  fast,  on-line,  dynamic  site  model  registration  of 
IR  image  sequences  to  existing  site  models.  This  effort  built  upon  the  site  model  construction  capa¬ 
bility  described  above.  The  dynamic  image  to  site  model  registration  approach  involves  the  render¬ 
ing  of  a  reference  image  from  the  site  model  to  the  approximate  viewing  perspective  of  the  sensor. 
Then  the  sensed  image  is  registered  to  the  rendered  reference  image.  The  dynamic  registration  may 
be  used  in  the  terminal  guidance  of  a  weapon  against  a  fixed  land  target,  or  may  be  used  to  provide 
rapid  updates  to  existing  site  models  with  live  sensor  imagery.  Details  of  this  effort  are  given  in  [5]. 


3 


2.3  McDonnell  Douglas 


2.3.1  Extraction  of  Higher  Order  Target  /  Clutter  Statistics  Using  Wavelets 

McDonnell  Douglas  investigated  the  use  of  wavelet  subbands  for  extraction  of  higher  order  statistics 
from  sensor  data,  for  potential  use  in  the  detection  of  targets  in  clutter.  This  investigation  was  con¬ 
ducted  in  the  context  of  MDA’s  General  Pattern  Match  (GPM)  ATR  algorithm,  which  bases  its  tar¬ 
get  decisions  on  likelihood  scores  computed  from  zeroth  order  pixel  statistics  (histograms),  extracted 
within  regions  of  a  hypothesized  target  templates  applied  to  the  sensed  image  data.  Because  the 
zeroth  order  analysis  is  invariant  to  spatial  redistributions  of  pixels  within  a  region,  this  approach  is 
insensitive  to  texture  within  a  region,  to  the  internal  structure  of  a  region,  and  to  clutter  characteris¬ 
tics. 

Because  joint  higher  order  distributions  are  computationally  prohibitive  and  also  would  be  too 
sparsely  populated  to  produce  a  meaningful  statistic,  we  investigated  the  use  of  wavelet  transforms  to 
reduce  the  joint  random  distribution  to  a  distribution  of  a  single  new  random  variable.  A  2-D 
wavelet  transform  was  applied  to  sensed  target  imagery,  and  the  resulting  subbands  were  processed 
with  the  GPM  algorithm,  using  a  generic  target  detection  template  (used  for  detecting  mobile  targets 
in  clutter).  Unfortunately,  the  use  of  wavelet  subbands  to  extract  higher  order  statistics  did  not  im¬ 
prove  performance  beyond  that  obtained  using  zero  order  statistics.  This  was  found  to  be  attribut¬ 
able  to  the  interaction  of  wavelet  filters  with  the  target  boundaries.  A  more  detailed  description  of 
this  investigation  and  the  results  can  be  found  in  [3]. 

2.3.2  Use  of  Wavelets  to  Extract  Structural  Features  from  Images 

McDonnell  Douglas  investigated  the  use  of  wavelets  to  extract  local  feature  points  to  be  used  for 
matching  2-D  and  3-D  structure  in  multiple  images.  Feature  values  were  computed  at  each  pixel 
location  as  a  function  of  wavelet  coefficients  associated  with  the  location.  The  feature  values  were 
thresholded  to  isolate  dominant  feature  points.  Where  feature  points  were  clustered,  the  cluster  of 
feature  points  was  replaced  by  the  centroid  of  the  cluster.  Corresponding  points  in  consecutive 
images  of  a  FLIR  image  sequence  were  successfully  identified  using  a  local  search  technique.  De¬ 
tails  of  this  effort  are  given  in  [7]. 

3.  Publications  and  Technical  Reports 

The  publications,  technical  reports,  and  interim  progress  reports  produced  under  this  contract  are 
listed  in  the  References  section.  In  addition  to  these  papers,  Summus,  Ltd.  is  preparing  two  papers 
based  on  their  work  under  this  contract.  Manuscripts  and  reprints  of  theses  papers  will  be  submitted 
when  they  are  available. 

4.  Scientific  Personnel 

McDonnell  Douglas  Aerospace:  Rich  Peer,  Jim  Meany,  Steve  Schwartz,  Chris  Martens,  Steve 
Wright 

Summus.  Ltd.:  Bjorn  Jawerth 
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University  of  Maryland  Rama  Chellappa,  Qinfen  Zheng,  Chandra  Shekhar,  Philippe  Burlina,  Azriel 

Rosenfeld 

Honors.  Awards,  and  Degrees:  None.  Participants  are  post-docs,  faculty,  and  industry  personnel. 

5.  Reportable  Inventions 

None  of  the  software  or  algorithms  developed  on  this  program  were  considered  to  be  reportable  in¬ 
ventions. 
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